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ABSTRACT 

The report documents activities and findings of the 
Preschool Evaluation Project which is developing a model for 
evaluating program provision to handicapped preschoolers and creating 
a longitudinal data base to track the short- and long-term progress 
of preschool children receiving special services in Montgomery 
County, Maryland. During the first year (1983-84) of the data base 
development phase the project conducted the following data collection 
activities: testing of 123 handicapped preschoolers with the Battelle 
Developmental Inventory (BDI); documentation of type and quantity of 
service provided; survey of parents regarding degree of satisfaction 
with their child's special education program. Major findings included 
the following: (1) with the exception of gross motor skills, Project 
children exhibited significant improvements in all areas of 
development beyond that expected by maturation alone; (2) the largest 
gains were in the areas of personal-social and cognitive skills; (3) 
children labeled multihandicapped made significant gains in all areas 
but motor skills; (4) children labeled "articulation disordered" or 
"language disordered" made gains but not in speech and language 
areas; (5) parents reported overwhelming satisfaction with their 
preschool child's program; (6) service providers did not use BDI test 
results in educational planning or discussions with parents. The body 
of the report details methodology, results, the parent satisfaction 
survey, and supplementary BDI validation studies. Appendixes include 
the parents' evaluation questionnaire, the BDI questionnaire, and 
additional information about the "value-added analysis" approach 
employed in the study methodology. (DB) 
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EXECUTIVE SUMMARY 



PRESCHOOL EVALUATION REPORT: 
YEAR 1 REPORT 

In October, 1980, the Preschool Evaluation Project was funded by the 
Maryland Department of Education to develop a model for evaluating several 
aspects of program provision to handicapped preschoolers. An outgrowth of 
this project was the recognition of a need for collection of long-term data 
on a large number of handicapped preschoolers. This need became apparent 
when co»aparisons of beginning and end-of-year test scores were made. 
Although the comparisons proved interesting, interpreting the scores was 
difficult because of various problems with the different assessments used 
and because of the short amount of time between pre- and posttest. 

As a result, the second phase of the Preschool Evaluation Project began in 
September, 1983. Phase 2, like Phase 1, has been jointly administered by 
two departments in MCPS: the Department of Special Education and Related 
Services and the Department of Educational Accountability. The purpose of 
the project is to create a longitudinal data base which can be used to 
document the short- and long-term progress of handicapped children who 
receive special services as preschoolers in Montgomery County. 
The following three data collection activities were conducted during the 
first year of the Project; 

1. Handicapped preschoolers were tested with the Battelle 
Developmental Inventory (BDI) prior to entry into special services 
and at the end of the school year. 

2. The type and quantity of service that each child received were 
documented. 

3. Parents were surveyed regarding the degree of satisfaction with 
the services provided by their child's special education program. 

In addition, because the Project was using the BDI, a new assessment 
instrument, two studies were conducted to examine the appropriateness of 
this tes t. The BDI was no t ye t commercially available when it was 
incorporated into the Project. The objective of the first study was to 
determine how MCPS service providers felt about the validity of the BDI 
based on the results for the child(ren) they served. The second study 
involved a comparison of parent and teacher information about the child's 
performance on some of the BDI Items. Some of the items are scored by 
asking an informant whether the child can do certain things. Several of the 
items in the test focus on the child's performance in a structured, school- 
like setting, and concerns were raised regarding whether teachers rather 
than parents should serve as informants for these items. 
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THE PROJECT POPULATION 



Beginning in September, 1983, and ongoing throughout the year, all preschool 
children (ages birth to five years) who had been diagnosed as handicapped 
and were new to special services in Montgomery County Public Schools (MCPS) 
were identified for potential inclusion in the project. During the 1985-84 
school year, 123 preschool children who began receiving special services 
through Montgomery County Public Schools were pre- and posttested, using the 
Battelle Developmental Inventory. Characteristics of these children were as 
follows: 

o Mean age at time of pretest was AO months (range: 4-64 months). 

o Racial/ethnic makeup of the group was 72 percent white, 20 
percent black, 5 percent Hispanic, and 3 percent Asian. 

o At least 20 percent of the children were from low-income families 
as measured by participation in Head Start. 

o "Language disordered" was the most frequent handicapping 
condition (43%), with mul tihandicapped the next most frequent 
subgroup (24%). 

o Sixty-three percent of the children were enrolled in public school 
programs. Of those attending private programs, 27 percent were 
enrolled in a program serving children with primary speech and/or 
language impairments. 

o Most children required a high level of intervention, with 64 
percent in Level 4 or 5 programs. 

o Three- four ths of the children received speech and language 
therapy; the average amount of time per week was one hour. 

o Children labeled as multihandicapped received a similar amount of 
speech and language therapy but more occupational and physical 
therapy than children with other handicapping conditions. 

FINDINGS 

One of the major goals of the Preschool Evaluation Project was to address 
the issue of efficacy of the intervention which these preschool handicapped 
children received. The findings with regard to efficacy were the following: 

o With the exception of gross motor skills, Project children 
exhibited statistically significant improvements in all areas of 
development beyond what would have been expected by maturation 
alone. 

o The largest gains due to intervention were in the areas of 
personal-social and cognitive skills. 

o Children labeled as multihandicapped exhibited statistically 
significant gains attributable to intervention in all areas of 
development except motor skills. 
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o Children labeled as "articulation disordered" or "language 
disordered" made gains attributable to intervention, but not in 
speech and language areas. 

o An analysis of specific factors (e.g., number of minutes in 
therapy, race, and handicapping condi tion) which might be 
associated with progress or predictive of gain due to intervention 
showed no consistent trends. 

Other findings from the Project were: 

o Parents reported overwhelming satisfaction with their preschool 
child's program and related services. 

o Service providers of Project children reportedly did not use BDI 
test results in educational planning or discussions with parents 
about their child's educational progress. However, the majority 
(over 70%) considered BDI results as within plus or minus three 
months of their estimate of the child's functioning. 

o Reliance on parents as informants for BDI items related to school 
or group activities, by and large resulted in responses similar to 
those provided by the child's teacher. 

These findings indicate that the gains these preschool handicapped children 
made during the 1983-84 school year were attributable to the services they 
received and were greater than that which would have been expected with 
maturation alone. These same children, along with new children, are being 
tested in the second year of the Project. Data from Year 2 of the Project 
will provide even more comprehensive information about the efficacy of 
intervention because of an increased number of children in the Project 
population, a greater number of children in each handicapping classifica- 
tion, more extensive data on services received, and a longer time between 
pre- and posttesting. 
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INTRODUCTION 



In October, 1980, the Preschool Evaluation Project was funded by the 
Maryland Department of Educatiun to develop a model for evaluating several 
aspects of program provision to handicapped preschoolers. An outgrowth of 
this project was the recognition of a need for collection of long-term data 
on a large number of handicapped preschoolers. This need became apparent 
when comparisons of beginning and end-of-year test scores were made. 
Although the comparisons proved interesting, interpreting the scores was 
difficult because of various problems with the different assessments used 
and because of the short amount of time between pre- and posttest* 

As a result, the second phase of the Preschool Evaluation Project began in 
September, 1983. Phase 2, like Phase 1, has been jointly administered by 
ttfo departments in MCPS: the Department of Special Education and Related 
Services and the Department of Educational Accountability. The purpose of 
the project is to document the short- and long-term progress of handicapped 
children who receive special services as preschoolers in Montgomery County. 

The following factors were considered in designing a process for documenting 
progress: 

o An objective, uniform assessment tool which could serve as a valid 
and reliable measure of functioning was needed. The instrument 
or instruments had to assess functioning in a number of 
developmental areas to provide a comprehensive picture of the 
child's strengths and weaknesses. The Battelle Developmental 
Inventory (BDI) was selected as the assessment device. 

o Baseline data, or preintervention functioning, had to be 
collected to ascertain the entry-level characteristics of the 
population under investigation and to allow conclusions regarding 
the effects of intervention. That is, to discuss how far a child 
has come, one must have an accurate picture of from where he or 
she began. 

o Ongoing, periodic reassessment with the same instrument as 
utilized to obtain preintervention data was necessary to document 
progress. If different test instruments were used, changes in 
ability level could be attributed to differences In test 
construction, content, and level of difficulty, rather than to 
actual child change. 

o Information about type and quantity of special education services 
provided was needed to allow relationships between gains and 
services "o be explored. 

o Information about parent satisfaction was necessary to provide 
information for program improvement and to relate child gains to 
parent satisfaction. 

The following three data collection activities conducted during the first 
year of the Project were related to the objective of documentation of 
progress : 



1. Testing handicapped preschoolers with the Battelle Developmental 
Inventory (BDI) prior to entry into special services and at the 
end of the school year. 

2. Documentation of the type and quantity of service that each child 
receives. 

3. Surveying parents regarding the degree of satisfaction with the 
services provided by their child's special education program. 

In addition, because the Project was using the BDI, a new assessment 
instrument, two studies were conducted to examine the appropriateness of 
this test. The BDI was not yet commercially available when it was 
incorporated into the Project. Thus, service providers within MCPS were not 
familiar with, and were frequently skeptical of, this assessment tool. It 
was considered critical to obtain a "reading" of their opinion of the test, 
based on the results for the child(ren) they served. The second study 
involved a comparison of parent and teacher information about the child's 
performance on some of the BDI items. Some of the items are scored by 
asking an informant whether the child can do certain things. Several of the 
items in the test focus on the child's performance in a structured, school- 
like setting, and concerns were raised regarding whether teachers rather 
than parents should sarve as informants for these items. 

This report will present the results of the first year of Phase 2 of the 
Preschool Evaluation Project (1983-84). 



METHODOLOGY 

SAMPLE 

Beginning in September, 1983 and ongoing throughout the year, all preschool 
children (ages birth to five years) who had been diagnosed as handicapped 
and were new to special services in Montgomery County Public Schools (MCPS) 
were identified for potential inclusion in the project. An introductory 
letter from the director of the Department of Special Education and Related 
Services was mailed to parents explaining the Project and requesting 
permission to test their children. Subsequently, Project staff contacted 
the families regarding scheduling of nesting. Cf the 278 families contacted 
during the first year of the Project, only six refused to participate. The 
most frequent reason for noncompliance was the parents 1 reluctance to have 
additional testing done. One hundred and fifty children were pretested 
between October, 1983, and January, 198A, an(* 124 of them received 
posttesting in May and June, 1984. The reasons for lack of posttesting were 
parental refusal (N=4), moved out of county (N=9), and child dropped because 
Project learned child had previously received services (N=13). Children 
pretested after January, 1984, (N=122) were not included in the posttesting 
because of an insufficient amount of time between tests. Ihese children will 
be posttested at the end of the 1984-85 school year. Additional information 
regarding the population who received pre- and posttesting will be provided 
in the Results section. 
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The Project continues to test all preschool children new to services in 
MCPS. However, only the first year's data are presented here, 

CHILD ASSESSMENT 

The Battelle Developmental Inventory (BDI) was selected to measure 
longitudinal progress in preschool children receiviug special services 
through Montgomery County Public Schools (MCPS) for the following reasons: 

o It is appropriate for use with children from birth to eight years. 

o Functioning is assessed in five areas of development (called 
"domains") - , personal/ social, adaptive, motor, communication, and 
cognitive. 

o Normative data are provided, based on approximately 800 children 
from all sections of the United States. 

o Adaptations for certain handicapping conditions are built into the 
test administration. 

Administration of the BDI involved two approaches. The Personal-Social and 
Adaptive Domains, which focus on the child's skills in interacting with 
peers and adults and in self-help areas, were administered in an interview 
format, either face-to-face or by phone, with the parent serving as 
informant. The other three domains contained structured items, which were 
administered directly to the child in a specified format. The Motor Domain 
assesses the child's abilities in both gross and fj.ne motor skills. The 
Communication Domain evaluates a child's understanding and use of language, 
i.e., both receptive and expressive skills. Finally, the Cognitive Domain 
assesses a child' o general knowledge of the world. Although the BDI allows 
scoring of some items based on observation, this method was not generally 
employed in collecting test data because of time constraints. 

Testing was done by nine part-time employees, Most of testers had advanced 
degrees in early childhood or special education, and all had prior 
experience testing and working with infants and preschoolers with 
developmental disabilities. Total time involved in assessment averaged two 
and one-half hours per child (30 minutes for the parent interview, two 40- 
minute testing sessions with the child, and 15 minutes for dcoring). 
Testing was done within the child's program or at home, depending on the 
preferences of the parents and the teachers. 

To document progress related to the receipt of special services, pretesting 
was to have occurred prior co a child's entry into these special services. 
During the first year of the Project, however, pretesting was done in the 
first few months after a child's enrollment in special services. This delay 
in pretesting was the result of difficulties in obtaining the BDI which had 
not yet been released for publication. Since July, 1984, almost all Project 
children have been tested prior to program entry. 

All posttesting occurred during May and June of 1984. As with pretesting, 
testing occurred either in the child's home or program. Efforts were made 
to assign each child to the same tester for both the pre- and posttest- 
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Both pre- and posttest results were sent to parents, and a copy of the test 
booklet with scores was sent to each child ! s service provider. 



DOCUMENTATION OF SPECIAL SERVICES 

Identification of services was done once during the first year of the 
Project and will be done three times in Year 2 and Year 3, Information 
regarding type and amount of services scheduled for children in the Project 
was collected by contacting each child's teacher or" primary service 
provider. * 

The form with the scheduled program time and related services was sent to 
each child f s teacher or primary service provider to verify that the child 
actually received the services he or she was scheduled to receive. 
Service providers verified informa tion for a one-week period in the spring 
of 1984. The form used for collecting this Information is included in 
Appendix A. For purposes of analysis of Year 1 data, only scheduled, rather 
than received, services were utilized. Given that service verification 
informa tion was only collected for a one-week period over the entire year, 
it was felt that the scheduled information was the better source for 
services. The Year 2 report will include information on scheduled and 
received services for three two-week periods. 

PARENTAL SATISFACTION WITH SERVICE PROVISION 

During Phase 1 of the Preschool Evaluation Project (1980-82), a 
questionnaire was developed to assess parental satisfaction with the 
services received by their child during a specified school year. The 
questionnaire asked the parents to rate their satisfaction with the overall 
program and the amount and quality of several different types of services 
and to indicate the frequency of home visits. A copy of the questionnaire 
is included in Appendix B. 

At the end of the 1983-84 school year, parents of the Project children were 
mailed a questionnaire which they completed and returned. One hunared and 
thirty-one questionnaires were mailed, and 55 were returned (42%). 

BDI VALIDATION 

As noted previously, the BDI had not yet become commercially available when 
it was incorporated into the Project in the fall of 1984. Thus, service 
providers within LCPS were not familiar with this assessment tool. The 
first of the two BDI studies was undertaken to obtain feedback on the 
usefulness of the test results and service provider impressions of BDI 



1. Information on special education program placement and related services 
is included on the Computerized Educational Data System (CEDS), a 
computerized system used in Montgomery County for accessing data regarding 
both handicapped and nonhandicapped children. However, for preschoolers, and 
particularly those in private programs, the only information included on 
CEDS is program and not-related services. Therefore, an alternative approach 
to information gathering was chosen. 




results in each domain. At the end of the 1983-84 school year, 
questionnaires were sent to 131 service providers who had received the 
results of at least one BDI test. A copy of the questionnaire is included in 
Appendix C. 

The second study involved a comparison of information supplied by the 
parent to that supplied by the child's teacher and was based on a concern 
about the accuracy of parent responses to some of the items. Discussions 
were held with Project testers to identify which test items parents 
appeared to have little experience observing and which teachers might 
observe regularly. E'.ght items were selected and are presented in Table 16. 
At the time of posttesting, testers administered the selected items to the 
parent and, subsequently, administered the same items to the child's teacher 
or service provider. The responses of the parent and the teacher to those 
eight items were compared. For each item, comparisons were available on a 
different number of children, ranging from 37 to 71. 



RESULTS 

DESCRIPTIVE INFORMATION ON THE CHILDREN IN THE STUDY 

Number, Age, Sex, Race, and Income . During the 1983-84 school year, 124 
preschool children who began receiving special services through the 
Montgomery County Public Schools were pre- and posttested with the BDI. 
Because one child was missing some demographic data, only 123 children were 
utilized in the data analysis. The mean age in months at the time of 
initial testing was 40 months (range: 4-6i months). Table 1 presents the 
number of children in the various age categories. At the time of pretest, 
most children (83%) were above two years of age. Eighty-two (67%) were 
boys. The racial/ethnic makeup of the group was 88 whites (72%), 25 blacks 
(20%), 6 Hispanics (5%), and 4 Asians (3%). Figure 1 presents the 
characteristics of the children at the time of the pretest. 

Data regarding parental education and occupation will be available at the 
end of Year 2 to provide an index of socioeconomic status (SES). However, 
this information is being collected via record review and was not available 
when the other data were analyzed. To obtain a gross indication of SES, 
subjects were divided into two categories: those who had attended Head 
Start and those who did not. This criterion was selected because the Head 
Start program primarily enrolls children from low-income families. Thus, 
with 25 subjects attending Head Start, at least 20 percent of the Project 
population was from low income families. 

Handicapping Condi tion . The handicapping conditions of the subjects were 
obtained, using group codes frcm the Computerized Educational Database 
System (CEDS), and are presented in Table 2. Children with language 
disorders comprised the greatest portion of the population (43%) with 
multiply handicapped children the next most frequent subgroup (24%). Data 
related to handicapping condition and special services are presented 
graphically in Figure 2. 



2. Not all items were administered to all parents/ teachers, because of the 
varying developmental levels of the Project children. 

ERIC 5 12 



TABLE 1 

Age in Months at Time of Initial Testing with BDI 



Age in Months N % 



0-23 21 17 

24-47 49 40 

48+ 53 43 



TABLE 2 
Handicapping Condition 



Primary Handicapping Condition N % 



Articulation Disorder 


IS 


15 


Language Disorder 


53 


43 


Voice Disorder 


1 


1 


Hearing impairment 


6 


5 


Visual impairment 


7 


7 


Orthopedic impairment 


1 


1 


Multiply handicapped 


30 


24 


Nonhandicapped, receiving 






services a 


5 


4 


Handicapping condition net 






specified 0 


2 


2 



a, These five children had as their "handicapping condition" two CEDS group 
codes which indicated a nonhandicapped group code, either "nonhandicapped 
speech and language" or "special need noncategor* cal," 

b. Other available information indicated these children were enrolled in 
programs and were receiving itinerant speech and language services, but 
there were no data regarding their handicapping condition on CEDS. 
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FIGURE 1 

Characteristics of Children at Pretest 
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Age ot Pretest 



FIGURE 2 

Handicapping Condition and Service ■ Information 
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Program . The 123 Project children were enrolled in ten different Montgomery 
County programs providing special services to handicapped children. These 
programs varied in scope and intensity of service, the type of child 
generally served, and sponsorship. Four of the programs were private; 
however, tuition for all Project children in these programs was paid for by 
MCPS. One of these programs tended to serve children with primary speech 
and/or language impairments, whereas the other three served children wi th 
significant developmental disabilities in more than one area. The remaining 
six were public school programs. Four served specific populations (e.g., 
preschool children with visual, auditory, or speech/language disorders), 
whereas the other two served children with more pervasive developmental 
disabilities. 

The number of Project children enrolled in each of the ten programs during 
1983-84 is presented in Table 3. Sixty-three percent of the children (N=77) 
were enrolled in public school programs. Of the children attending MCPS 
programs, approximately one- third were enrolled in Program 7 (Head Start). 
It should be noted that these children were involved in the Project because 
of a speech and/or language impairment and were only receiving itinerant 
services. Head Start was not considered a special education program. Of 
the 46 children attending private programs which served children with 
primary speech and/or language impairments, 27 percent (N=33) were enrolled. 

Type and level of service . Within MCPS, children receiving special 
education are assigned a level of service, which generally indicates 
intensity. Lower-level services (Levels 1, 2, and 3) are consultative, 
itinerant, or resource; whereas more intense levels of service (Levels 4 and 
5) are found in special class settings or programs. Forty-four of the 123 
children (36%) were receiving Level 2 services, whereas the remaining were 
enrolled in Level 4 or 5 programs. Such findings suggest that this 
population on the whole was significantly impaired and required a high level 
of support and intervention. 

Preschool children, because of their age and/or disability, are sometimes 
served in the home. Eight Project children (7%) received some services at 
home, and .3 (57%) were enrolled in a center-based program. The remaining 
children received itinerant services at a school. The amount of program 
time provided to children in home- based and center- based programs is shown 
in Table 4. Children in home-based services received approximately 60 
minutes of program time each week. In contrast, almost half of the children 
who attended a center-based program received six to twelve hours weekly. 
Children receiving only itinerant services (e.g., PT, 0T 3 speech) were 
categorized for Project purposes as receiving only related services and no 
program time. 

Information on time scheduled in speech, physical, and occupational therapy 
services (both itinerant and program based) was available for 120 children. 
The number of children receiving each of these services and the mean number 
of minutes per week in services are presented in Table 5 and illustrated in 
Figure 3. More than three-fourths of the Project children received speech 
and language therapy during 1983-84, whereas physical or occupational 
therapy was provided to significantly fewer children. In addition, the 
amount of speech and/ or language therapy provided on a weekly basis was much 
higher than physical or occupational therapy services. 
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TABLE 3 
Program Enrollment 



Program 




N 




% 


LNunpu U± JLC 










1 (Child Pan tor} 




A 
\j 




5 


9 ^Thrl Phnmh Phi 1 H fpnt*pr} 
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I 




1 


3 (Montgomery Presch, Achievem, Ctr. 


) 


5 




A 


H \iTca UIUcUI V-»cll Let s ) 




33 




27 


Public 










5 (Auditory) 




6 




5 


6 (Early Childhood Program) 




A 




3 


7 (Head Start) 




• 25 




20 


8 (Preschool Education Program) 




16 




13 


9 (Itinerant Speech/Language) 




19 




15 


10 (Vision) 




7 




6 


TABLE 


A 








Minutes in 


Program 






Minu tes Home-based 




Center- based 




per week (N=8) 




(N=70) 






N % 




N 


% 




1-89 5 63 




0 






90-360 3 38 




21 


30 




361-750 0 




33 


47 




750+ 0 




16 


23 





17 

° in 
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TABLE 5 
Special Services Received 



Type of Service 



N 



Receiving 
Service 



Data for Those 
Receiving Services 
(Minutes/Week) 
Mean Range 



Speech and/or language 92 77 56 10-120 
therapy 

Physical therapy 26 22 54 10-120 

Occupational therapy 39 33 39 10-110 



a. Percentage of 120 children with information on services. 
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FIGURE 3 

Percentage of Children Receiving Related Services 
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It is logical to assume that children with a handicapping classification of 
"multihandicapped" have multiple needs and subsequently would be more likely 
than other handicapping conditions to receive varied services at a high 
level* Table 6 presents information regarding the number of multi- 
handicapped Project children (MH) receiving related services in 1983-84, as 
compared to those with other labels. This comparison is presented 
graphically in Figure 4. When compared to non-MH children, MH children 
received similar amounts of speech and language therapy during 1983-84; but 
children labeled as MH were more likely to be receiving physical and 
occupational therapy. 

EFFECTIVENESS OF INTERVENTION 

Approach . There are numerous questions which need to be addressed when 
studying the efficacy of early intervention, a few of them being the 
following: 

o What constitutes efficacy? Is it movement to a less intense level 
of service? Is it movement totally out of special education? Is 
it improved test scores in areas which have been the focus of 
intervention? 

o What is the appropriate time to assess efficacy? After one year in 
special education? After five yesrs? 

o Is it possible that preschool intervention is "effective" for 
children with some types of handicapping conditions, but not for 
others? 

For the purposes of this Project, several "stances" were taken regarding 
these issues. Program effectiveness for the first year of the Project was 
measured by the amount of gains in BDI test scores, without consideration of 
the need for fewer services. The longitudinal nature of this Project 
provides an extended period of time in which to assess and determine 
efficacy of services. Finally, as the number of children with each 
handicapping condition increases within the Project population, analyses of 
subgroup patterns of progress and response to intervention w/ill be possible. 

An additional problem is how to determine the amount of growth which should 
be attributed to program participation. Federal and state laws mandate free 
and appropriate services to children in Maryland with special needs cr 
handicaps. Thus, it is difficult, a well as illegal, to locate a sizable 
group of preschool handicapped children who are not receiving services to 
serve as a control or nontreatment group in studies of the efficacy of or 
response to Intervention. 

An alternative approach to looking at this issue is to employ a statistical 
procedure which compares .projected growth, based Ou preintervention 
functioning, with actual growth, based on the difference between pre- and 
pos tintervention scores. Gains attributable to intervention would be 
reflected in the differences between projected and actual growth. One such 
statistical strategy, value-added analysis (Bryk and Weisberg, 1976), was 
the method selected to analyze the changes in BDI test scores for children 
in this Project. 
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TABLE 6 



Related Services Received by Multlhandlcapped and Other Children 



Multlhandlcapped Other Labels 

(N=30) (N=93) 



Amount* r*>f Ueeklv 
Service In Minutes 


PT 


OT 


S/L 


PT 


OT 


S/L 








Percentage 






No service 


23 


30 


20 


94 


81 


24 


10-30 


30 


37 


23 


0 


4 


15 


31-60 


30 


33 


43 


2 


11 


44 


61+ 


17 


0 


13 


1 


1 


14 


Inf ormatlon 
not available 


0 


0 


0 


3 


3 


3 


Note. Table entries 


are 


column 


percentages . 
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FIGURE 4 

Comparison of Services Received by MultihandJ. :apped Children and Children 

Ll with Other Handicaps 
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With reference to interpreting the Project data on efficacy, the following 
are caveats which must be made: 

o Because of the delay in obtaining the BDI, the amount of time 
between pre- and posttesting was not as great as desired (Mean: 
4.5 months). This short amount of time means that there may not 
have been sufficient time for statistically significant progress 
to have occurred. Interpretation of results must take th?s factor 
into account, particularly with reference to certain populations 
for whom observable progress within a four-month period might be 
unrealistic. 

o Most children received some MCPS services prior to assessment 
with "he BPI for reasons noted earlier. The time the children 
were in MCPS services prior to testing varied somewhat, up to a 
maximum of approximately three and one-half months. Thus, progress 
made during that period would not have been recorded by uhe BDI. 
Certain handicapping .conditions may be associated with early and 
rapid progress upon intervention, with a subsequent leveling off. 
If this were to be the case, such children would not show as great 
an intervention effect in the analysis of the first-year data. 
Both of these problems will be corrected when second-year data are 
analyzed. 

o In interpreting group trends related to progress, it is important 
to consider the heterogeneity of the Project population and tha 
ranges of severity within a given handicapping population. 
Although there is a strong desire to ascertain the efficacy of 
early intervention with the "preschool handicapped population," 
this term includes a varied population who may exhibit quite 
different trends from the group as a whole. While this problem 
could be addressed by analyzing growth for subgroups, at this time 
the size of the Project subgroups for all but the most frequent 
handicapping conditions are too small to warrant analysis. Again, 
this problem will become less serious as more children become part 
of the Project. 

o Due to resource limitations, no data could be collected on the 
precise characteristics of the intervention or services provided 
to Project children (e.g., a cognitively oriented program, 
neurodevelopmental therapy, and pragmatically based language 
therapy). Thus, no comment can be made about the type or approach 
which is most effective with a particular group. 

Assessment Results . Mean BDI pre- and posttest raw scores and standard 
deviations by age for the 123 Project children are presented in Table 7. 
At the time of pretest, mean scores were below normal in development for 
ages. Mean posttest raw scores increased in all BDI domains. 

Several questions were asked to address the issue of child change as a 
function of intervention. All analyses were based on an N of 120 and were 
generated using the value-added strategy discussed earlier. Additional 
technical information on value-added analysis is presented in Appendix D. 
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TABLE 7 
BDI Scores by Age Groups 



ERIC 



BDI Domain Age in Months at Pretest 

0-11 12-23 24-35 36-47 48+ 

(N=8) (N=13) (N=18) (N=31) (N=53) 



Pretest Age 

Mean 9 15 31 42 53 

SD 2 2 4 4 4 

Personal-Social 
Pretest 

Mean 26 38 74 99 128 

SD 6 17 18 19 26 

Posttest 

Mean 38 51 110 125 142 

SD 7 19 31 18 25 

Normal Raw 

Score for 34-36 53-55 99-100 125-126 144-145 

Mean Age a 

Adaptive 
Pretest 

Mean 16 27 53 68 81 

SD 5 11 10 8 14 

Posttest 

Mean 27 36 70 79 87 

SD 7 13 15 10 14 

Normal Raw 

Score for 26-27 38-39 65 80 91 

Mean Age a 

Gross Motor 
Pretest 

Mean 9 19 48 59 65 

SD 4 10 12 8 13 

Posttest 

Mean 12 28 54 62 70 

SD 6 15 12 9 14 

Normal Raw 

Score for 27-29 40-41 57 64 71 



Mean Age 



a 



17 



24 



TABLE 7 (cont.) 
BDI Scores by Age Groups 



BDI Domain Age in Months at Pretest 

0-11 12-23 24-35 36-47 48+ 

(N=8) (N=13) (N=18) (N=31) (N=53) 



Fine Motor 

Pretest 

Mean 6 
SD 3 

Pos ttest 

Mean 10 
SD 5 

Normal Raw 

Score for 14 

Mean Age a 

Receptive Language 

Pre test 

Mean 7 
SD 2 

Pos ttest 

Mean 12 
SD 2 

Normal Raw 

Score for 9 

Mean Age a 

Expressive Language 

Pretest 

Mean 6 
SD 2 

Posttes t 

Mean 12 
SD 4 

Normal Raw 

Score for 9 

Mean A^e a 




13 30 41 49 

6 8 6 8 

18 36 48 55 

8 10 6 11 

20 34 45 56 

10 15 21 28 

2 3 6 9 

13 18 25 31 

4 5 6 9 

13 21 29 37 

9 18 28 36 
4 5 6 9 

14 25 34 40 
6 8 8 9 

16 31 40 47 



TABLE 7 (cont.) 
BDI Scores by Age Groups 



BDI Domain 




Age 


in Months at 


Pretest 






0-11 


12-23 


24-35 


36-47 


48+ 




(N=8) 


(N=13) 


(N=18) 


(N=31) 


(N=53) 


Cognitive 












Pretest 












Mean 


9 


15 


29 


44 


55 


SD 


4 


7 


7 


10 


17 


Posttest 












Mean 


17 


19 


39 


53 


70 


SD 


5 


8 


11 


12 


19 


Normal Raw 












Score for 


17 


22 


37 


52-53 


71-72 


Mean Age 3 













a. "Normal raw score" was taken from the BDI Age Equivalency tables. It is 
the raw score that corresponds to the mean pretest age equivalency for the 
group. For example, in the Personal-Social Domain, a child of 45 would be 
expected normally to get a raw score of 131. 
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In which areas of development (i.e., BDI domains) did Project children 
exhibit gains attributable to intervention? 

Nearly all Project children had higher scores on the posttest than on the 
pretest. However, it was critical to ascertain the relative contributions 
of intervention and normal growth to this change between pre and post- 
testing. Table 8 presents information about the extent of change between 
pre- and posttest, including pre- and posttest score differences, standard 
deviations, and the amount of gain or progress attributable to intervention 
(referred to as "v" for the value added by intervention) for the 123 Project 
children. A "v" of zero indicates that, in general, changes in BDI scores 
were the result of normal growth, lu other words, growth equaled exactly 
what was expected based on pretest scores. A negative "v" indicates that 
the posttest score is poorer than would have been predicted without 
intervention. A positive n v" reflects an intervention effect. Figure 5 
shows the actual pre- and posttest scores for each of the domains and the 
expected posttest score based on the value-added analysis. 

With the exception of gross motor skills, Project children exhibited 
significant improvements in all areas of development a", a function of 
participation in special services. The largest gains were seen in the 
Personal-Social and the Cognitive domains. The last column in Table 8 
presents the number of months gain that is equivalent to the observed change 
in raw score points. In Personal-Social, the children gained 4.4 months 
above what would nave been expected for them due to maturation alone. The 
corresponding figure in the cognitive area was 5.1 months gain. Based on 
these data, it appears that intervention was measurably effective for these 
children, even over the short time between pre- and posttests. 

How do gains compare for children with different handicapping conditions? 

As was evident in Table 2, certain handicapping conditions were more highly 
represen-ed than others in the Project sample. The most frequently 
occurring conditions were articulation disorders, language disorders, and 
multiple handicaps. The small numbers in other conditions did not allow 
their consideration for this question. The data on the extent of gain due 
to intervention for the three most frequently occurring handicapping 
conditions are shown in Table 9. 

Children labeled as mul tihandicapped (N=30) exhibited statistically 
significant treatment gains in all areas of development with the exception 
of fine and gross motor. Children labeled as "language disordered" (N=53) 
comprised the largest disability group in the Project population. These 
children exhibited gains attributable to intervention only in personal- 
social and cognitive. Finally, children labeled as "articulation 
disordered" (N=18) exhibited progress which was the result of intervention 
only in fine motor arid cognitive areas. 

All three groups of Handicapped children appear to have received some 
benefits from the special services received in Montgomery County. Despite 
what were probably quite different interventions, with regard to content and 
and amount, significant gains occurred in cognitive abilities in all three 
groups. 
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TABLE 8 



Amount of Change Between Pre- and Posttest 
Scores Attributable to Intervention (N=123) 



BDI Mean 


Amount Attributable to 


Intervention^ 


Domain Differences 






Between Pre- 




Equivalent 


and Posttests a 




to Months 




Mean S,D, p 


Gain 


lersonal- 20 


9-7 17.0 ,000 


4,4 



social 



Adaptive 


9 


2. A 


9.0 


.002 


1.8 


Gross motor 


6 


-0.5 


6.3 


NS 


0 


Fine motor 


5 


1.2 


5.4 


.01 


1.3 


Receptive 
language 


3 


1.1 


5.7 


.03 


1.8 


Expressive 
language 


5 


1.7 


5.6 


.001 


2.1 


Cognitive 


12 


6.6 


9.6 


.001 


5.1 



a. Raw score points 



b. Raw score gain which exceeded that expected by maturation alone. The 
difference between the first and second column, e.g., 20-9*7, was the amount 
of growth expected by maturation alone. 



FIGURE 5 

Actual and Expected Posttest Scores 
for Each BDI Domain 
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FIGURE 5 (Cont.) 
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FIGURE 5 (Cont.) 
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TABLE 9 

Amount of Change Between Pre- and Posttest 
Scores Attributable to Intervention 
by Specific Handicapping Conditions 



i 

BDI Mean 




Amount Attributable 


to Intervention* 5 


Between Pre- 
• and Post tests 


a 


Mean 


S.D. p 




Equivalent 
to Months 
Gain 


Articulation 
Disorder (N=18) 












Mean Age at 
Pre test: 
51 months 




• 








Personal- 
social 


9 


-2.8 


10.0 


N3 


-1.3 


Adaptive 




-1.1 


9.3 


NS 


-0.8 


Gross motor 


6 


0 


7.6 


NS 


0.1 


Fine motor 


9 


3.7 


5.2 


.01 


4.0 


Receptive 
language 


1 


-1.7 


7.5 


NS 


-2.7 


Expressive 
language 


3 


-1.2 


5.3 


NS 


-1.5 


Cognitive 


11 


5.8 


11.1 


.05 


4.5 



a. Raw score points 

b. Raw score gain which exceeded that expected by maturation alone. The 
difference between the first and second column, e.g., 9-3, was the amount 
of growth expected by maturation alone. 
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TABLE 9 (cont.) 



Amount of Change Between Pre- and Posttest 
Scores Attributable to Intervention 
by Specific Handicapping Conditions 



BDI Mean 

Between Pre- 
and Posttests 3 



Amount Attributable to Intervention 



Mean 



S.D. 



Equivalent 
to Months 
Gain 



Language 
Disorder (N=53) 



Mean Ag°. at 












Pretes t: 












44 months 












Personal- 


25 


13.3 


17.1 


,001 


6.0 


social 












Adaptive 


10 


2.2 


9.8 


NS 


1.7 


Gross motor 


5 


-1.2 


5.8 


NS 


-1.3 


Fine motor 


5 


0 


5.0 


NS 


0.4 


Receptive 












language 


3 


0 


5.3 


NS 


0.5 


Expressive 












language 


4 


1.3 


6.2 


NS 


1.6 


Cognitive 


12 


7.2 


9.0 


.001 


5.5 



a. Raw score points 

b. Raw score gain which exceeded that expected by maturation alone. The 
difference between the first and second column, e.g., 25-133, was the amount 
of growth expected by maturation alone. 
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TABLE 9 (cont.) 

Amount of Change between Pre- and Posttest 
Scores Attributable to Intervention 
by Specific Handicapping Conditions 



BDI Mean 

Between Pre- 
and Posttests 3 



Amount Attributable to Intervention 



Mean 



S.D. 



Equivalent 
to Months 
Gain 



Multi- 
Handicapped (N=30) 



Mean Age at 
Pretes t: 
29 months 












Personal- 
social 


18 


9.9 


20. 1 


.05 


4.5 


Adaptive 


10 


4.6 


7.7 


.01 


3.5 


Gross motor 


4 


0 


6.7 


NS 


-0.1 


Fine motor 


5 


1.4 


6.5 


NS 


1.5 


Recep tlve 
language 


4 


2.6 


4.7 


.01 


4.1 


Expressive 
language 


6 


3.2 


4.1 


.001 


4.0 


Cognitive 


10 


6.3 


8.9 


.001 


4.9 



a. Raw score points 

b. Raw score gain which exceeded that expected by maturation alone, The 
difference between the first and second column, e.g., 18-9.9, was the amount 
of growth expected by maturation alone. 
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The absence of treatment-related growth in motor skills in the- multi- 
handicapped children is of concern because this is an area of focus and 
treatment for many preschool handicapped children so labeled. This finding 
may be the result of the limited amount of time between pre- and post- 
testing or could be a weakness of the assessment tool. The absence of 
treatment-related growth in the articulation disordered group in expressive 
language is considered a reflection of limitations of the BDI. This test 
only minimally evaluates articulation/phonology, the area of deficit, and 
the focus of remediation for these children. Thus, the area in which these 
children would be most likely to exhibit treatment-related growth was not 
adequately evaluated. 

What factors were associated with the greatest gains in intervention? 

The presence and lack of substantive progress in intervention may be the 
product of myriad factors, events, or variables within and beyond the scope 
of any program. Multiple regression was used to explore the independent 
coutribution of a number of variables to the amount of progress attributable 
to intervention. Factors which were explored as possibly contributing to or 
hindering progress in special services were as follows: 

o Age in months at the time of initial testing 

o Race 

o Sex 

o Home-based services 

o Center-based services 

o Total number of scheduled minutes in a center-based program 

o Total number of scheduled minutes in therapy: speech- 
language, physical, and occupational 

o Handicapping condition 

o Income (defined as participation in Head Start) 

The regression analyses were based on an N of 120. The amount of variance 
explained and factors which were related to gains due to intervention for 
each BDI domain are presented in Table 10. The results indicated that 
factors examined explained only a small percentage of the variance in gains 
due to intervention. Furthermore, there was no consistent pattern from 
domain to domain as to the factors contributing to that amount of variance. 
The short time interval between pre- and posttest may have limited growth so 
that relationships between predictions and growth could not be detected. A 
better answer to the question of which factors predict gain due to 



3. For three of the 123 Project children, information was unavailable on 
their precise handicapping condition; and thus it was necessary to eliminate 
them from this analysis. 
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TABLE 10 



Results of Regression Analysis To Explain Gain Due to Intervention 

(N-120) 



Domain 


Percentage 

Variance 

Explained 


Significant Factors 3 


Direction of 
Factors 


Personal-social 


16 


Enrollment in a center-based program 
Number of minutes in speech therapy 
Number of minutes in physical thei^ t y 
Income 


+ 
+ 


Adaptive 


11 


Age in month a: time of pretest 
Income 


+ 


Gross motor 


12 


Being black (0=no, i=yes) 

Total number of program minutes/week 

Being speech/ language impaired (0=no, 1=> 


+ 

r es) 


Fine motor 


9 


Being black (0=no, l=yes) 
Income 




Receptive language 


13 


Number of minutes in speech therapy 
Number of minutes in physical therapy 
Being speech/language impaired (1 = yes) 




Expressive language 


20 


Enrollment in home-based services 
Enrollment in center-based services 
Number of minutes in physical therapy 




Cognitive 


13 


Age in months at time of pretest 

Being speech/ language impaired (1 = yes) 





a. Arranged in order of amount of variance explained. Gains were related to the 
higher values on the variables listed. 



b. For example, the children who made the most progress due to intervention in gross 
motor skills were black, received the least amount of Program time, and did not have 
a "speech/language impaired" handicapping classification. 
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intervention will be available in subsequent Project years when nhere are 
more children in the analyses and when the time in intervention j ill be 
longer. Also, analyses in subsequent years will include additional 
background data from the record review as well as attendance data, 

PARENT SATISFACTION 

Fifty-five of the 123 parents (45%) returned the questionnaire on parent 
satisfaction, although not all questionnaires had all items completed. 
Ninety-three percent of the respondents (N=51) indicated that they were 
satisfied or very satisfied with the amount of th^ overall program TIME, and 
95 percent (N=52) were similarly pleased with its quality. Satisfaction with 
the amount and quality of services is presented in Table 11, These results 
indicate overwhelming parent satisfaction with the preschool program and 
related services. However, parents appeared to be slightly less happy with 
the amount of counseling, home visits, and PT services provided. 

Thirty-eight of the 55 parents (69%) indicated they had received a home 
visit from a staff member in their child's program during the 1983-84 school 
year. Seventy-nine percent of the parents (N=30) received one or two 
visits. Of the 26 parents who provided information regarding the average 
length of the visit, 73 percent (N=19) reported home visits were one hour in 
length. The remaining parents indicated the visits were longer. 

To ascertain if there were particular special education programs for which 
parents indicated greater or lesser-satisfaction, questionnaire results were 
analyzed by program. Results are presented in Table 12, Again, parents, 
regardless of program, voiced a strong positive vote for the amount and 
quality of the program in which their preschool handicapped child was 
enrolled, although the numbers in manv of the programs were too small to 
allow definitive conclusions. 



BDI VALIDATION STUDIES 
OPINIONS OF SERVICE PROVIDERS 

Seven ty- three questionnaires on the BDI (65%) were returned by service 
providers. The type of individual completing the questionnaire and program 
affiliation are provided in Table 13, Over half the respondents were either 
teache rs or speech- language pathologists. Fifty-nine percent of the 
respondents based their answers on review or exposure to the results of the 
BDI for one to five children. 

Respondents were asked to indicate if they used BDI test results for 
educational planning and discussion with parents about their child's 
educational progress. Results, presented in Table 14, Indicate that most 
respondents did not utilize the BDI findings for either purpose, and for 
those who did, less than half noted the usefulness of these results as 
moderate or higher. These findings may be the result of several of the 
following factors: 
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TABLE 11 

Parental Satisfaction with the Amount and Quality of Services 



Service 




Percentage 
Satisfied or Very 
Satisfied with 
Aaount of Services 


N 


Percen tage 
Satisfied or Very 
Satisf ied'with 
Quality of Services 


Overall 
program 


55 


93 




55 


95 


Speech/ 
language 


50 


92 




48 


90 


OT 


34 


82 




32 


81 


PT 


28 


71 




27 


81 


Behavior 

management/ 

counseling 


25 


68 




26 


81 


Home visits 


24 


75 




25 


84 



a, N*s include only those parents whose children were receiving service 
and who completed the item. 



38 

31 




TABLE 12 

Parental Satisfaction by Program 



Program 


N 


Percentage 
Satisfied/Very 
Satisfied with 
Amount 


Percentage 
Satisfied/Very 
Satisfied with 
Quality 


1. 


(Auditory) 


3 


100 


100 


2. 


(Christ Church Child Center) 


0 






5 . 


vuniia Center; 


3 


100 


100 


4. 


(Early Childhood Program) 


3 


100 


100 


5. 


(Head Start) 


3 


100 


100 


6. 


(Montgomery Preschool 
Achievement Center) 


3 


100 


100 


7. 


(Preschool Education Program) 


10 


90 


80 


8. 


(Speech/Language) 


11 


73 


100 


9. 


(Treatment Centers) 


17 


100 


9A 


10. 


(Vision) 


2 


100 


100 
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TABLE 13 



Respondent? to BDI Questionnaire: Role and Program 
Affiliation (N-73) 







Percentage 


harac ter is tics 


N 


of Respondents 


Role 






Director/coordina tor 


0 





Physical therapist 


3 


h 


Special educator/ teacher 


26 


36 


Speech- language pa thologist 


20 


27 


Occupa tional the rap is t 


1 


1 


Head Start teacher 


8 


11 


Not specified 


15 


21 


Program 






1, (Auditory) 


2 


3 


2. (Christ Church Child Center) 


4 


5 


3, (Child Center) 


8 


11 


4. (Early Childhood Program) 


h 


5 


5« (Head Start) 


13 


18 


6, (Montgomery Preschool 






Ac*h i p vpmpn r (^pnTpv ) 




7 

/ 


7. (Preschool Education Program) 


8 


11 


8. (Speech/Language) 


17 


23 


9. (Treatment Centers) 


10 


1 h 


10. (Vision) 


2 


3 




TABLE 


14 


Use of BDI Test Results by 


MCPS Service Providers 






Percentage of Those 


Percentage 


Who Noted at Least 


Task N Who 


Use 


N Moderate Usefulness 



Educational 

planning 57 25 20 45 



Talking with 
parents about 
educational 

progress 57 21 6 30 



o Service providers received the test results after they had already 
made educational plans and decisions for the year, 

o Impressions were based on just pretest results, 

o Service providers may have viewed the BDI as one component of a 
research project that was not of clinical relevance, 

o Service providers may have been skeptical of the test and the 
testers and continued to rely on familiar, established assessment 
tools, 

o Speech- language pathologists might not have considered the BDI as 
detailed an assessment as other measures of speech and language, 
particularly in articulation. 

Service providers were asked to rate their impressions of the accuracy of 
the assessment in each of the five domains of the BDI, using a nine-point 
scale. A sizable percentage of the respondents (from 30% to 47%, depending 
on the domain) felt they could not make a judgment. Results, presented in 
Table 15, indicate that service providers who could make judgments viewed 
scores obtained on the Adaptive and Motor Domains as more reflective of 
their perceptions of the child's abilities than the scores obtained in other 
areas. The Communication Domain fared most poorly, with only slightly more 
than one- third considering the scores to be about right. However, for all 
domains, over 70 percent of those who felt they could make a judgment felt 
the BDI results were within plus or minus three months of their estimate of 
the child's functioning. Service providers who did not consider BDI test 
results an accurate representation of the child's abilities tended to 
consider the scores too low; i.e., these results tended to make the child 
look more impaired than the service provider judged him or her to be. 

As indicated previously, special educators and speech- language pathologists 
comprised a large portion of the respondents. Each of the two groups 1 
responses were analyzed to ascertain if certain professionals viewed the BDI 
in a particular way. Ratings of "accuracy*' of BDI scores varied between the 
two professional groups. Over three-fourths of the speech-language 
pathologists considered BDI scores accurate or within plus or minus three 
months (i.e., a rating of 2, 3, or 4) in all domains except Communication, 
for which only 60 percent considered the scores accurate. Eleven percent 
considered the scores six months lower than anticipated, and eleven percent 
considered the scores six months higher. Over half the special educators 
viewed BDI scores as "on target" or within plus or minus three months for 
all domains. The Adaptive Domain fared the best, with 80 percent of the 
special educators rating those scores as accurate. There was slightly less 
"enthusiasm" for the Personal-Social and Cognitive Domains, for which only 
55 percent considered the scores as accurate. 

Finally, service providers were asked to indicate on which of the following 
they formulated their impressions of BDI test results: clinical 
impressions/classroom performance, other test data, feedback from other 
professionals, and feedback from parents. Regardless of the domain in 
question, a majority of respondents indicated they most strongly relied on 
clinical impressions and/or classroom performance in judging the validity of 
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TABLE 15 

Rating of BDI Domains by Service Providers- 









Percen tage 


Rating BDI 


Score As: 






u o ma 1 n 


Mb 


1 


I 


3 


4 


5 


6 


7 


Personal- 
social 


51 


8 


12 


43 


16 


6 


10 


6 


Adaptive 


40 


0 


10 


55 


15 


10 


5 


5 


Motor 


43 


0 


7 


58 


16 


9 


7 


2 


Communication 


46 


7 


9 


39 


20 


17 


4 


4 


Cognitive 


44 


0 


2 


45 


22 


20 


0 


9 



a. Respondents were asked to rate their impressions of each of the BDI 
developmental domains, using the following scale: 



1 = BDI scores are six or more months higher than I think they should be, 

2 - BDI scores are about three months higher than I think they should be, 

3 - BDI scores are about what I think they should be, 

4 = BDI scores are about three months lover than I think they should be, 

5 = BDI scores are six or more months lower than I think they should be, 

6 = BDI scores are sometimes too high and sometimes too low, but 

generally about six months off. 
7= BDI scores are sometimes too high and sometimes too low, but 

generally about three months off. 
9=1 cannot make a judgment. 



b, N includes only those who felt they could make a 1-7 judgment. 
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BDI test results (Personal-social: 83 percent; Adaptive: 81 percent; Motor: 
57 percent; Communication: 67 percent; and Cognitive: 68 percent). 

PARENT-TEACHER COMPARISONS 

Seventy-eight parent- teacher comparisons on BDI items were available for 
aualysis. However, not all item comparisons were based on this number 
because not all children were administered all items. Item administration 
is determined by the child's functioning; therefore, an item notably above 
or below a child's ability was not administered. A score on an individual 
BDI item may be a "2" (correct; fully developed behavior), "1" (partially 
correct; emerging behavior), or "0" (failed item, no op per tuni ty, no 
response). Discrepancy scores were calculated for each of the eight items 
examined by subtracting the score given by the parent from the score given 
by the teacher. For example, if both parent and teacher indicated a child 
was unable to do a task (i.e., information from both informants resulted in 
a score of "0"), there was no discrepancy. If the teacher scored the 
child a "2" and the parent scored the child a "1," the discrepancy score 
was 1. 

Table 16 presents the eight items and the percentage of items with 
discrepancies between the responses of parent and teacher. Results 
indicated that for the majority of comparisons for most of the items no 
discrepancy occurred. This is particularly significant because these were 
items selected by the testers as those most likely to result in a 
discrepancy. PS61 ("serves as a leader in peer relationships") and A37 
("responds to instructions given in a small group and initiates an 
appropriate task without being reminded") were most likely to result In 
discrepant responses between parent and teacher. For the later item, one- 
fifth of the parent- teacher pairs gave responses which differed by two 
points. 

The direction of the differences between parents and teachers was determined 
and is presented in Table 17. In general, there was no strong trend of one 
group scoring the child higher than the other group, with the exception of 
Item PS44 regarding "show and tell" where the parents were more likely than 
the teachers to give the child a higher rating. 

In sum, it appears that reliance on parents as informants for information 
related to school or group activities by and large results in responses 
similar to those provided by the child's teacher. In those cases where it 
does not, there was no consistent trend for parents to over or 
underestimate their child's abilities in comparison to the teacher's 
estimate. 



SUMMARY 

The Preschool Evaluation Project was begun in September, 1983, to document 
short- and long-term progress of handicapped children who receive special 
services as preschoolers in Montgomery County. During the 1983-84 school 
year, 123 preschool children who began receiving special services through 
Montgomery County Public Schools were pre- and posttested, using the 
Battelle Developmental Inventory. Characteristics of these children were as 
follows: 
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TABLE 16 



Parent-Teacher Comparisons on 
BDI Interview Items 


Selected 




I torn 


N 


Percentage Responding: 
No dis- 1-point 2-point 
crepancy discrepancy discrepancy 


Demonstrates ability to 
"show and tell" without 
major discomfort (PS44) 


57 


81 


9 


10 


Attends to learning task 
or story in small group 
(A9) 


71 


73 


15 


12 


Follows rules given by 
adult for playing simple 
childhood games (PS63) 


37 


70 


11 


19 


Uses adults other than 
parents as resources 
(PS16) 


53 


68 


25 


8 


Completes learning tasks 
having two or more steps 
(A38) 


66 


65 


18 


17 


Follows classroom rules 
and directions (PS65) 


65 


60 


29 


11 


Responds to instructions 
given in small group and 
initiates appropriate 
task without being 
reminded (A37) 


68 


50 


29 


21 


Serves as leader in 
peer relationships 
(PS61) 


45 


42 


42 


16 



Note : Table entries are row percentages. 



TABLE 17 

Direction of Differences of Parent-Teacher Comparisons 



BDI Item Parent Higher No Teacher Higher 

Than Teacher Discrepancy than Parent 



Percentage 



PS44 


16 


81 


4 


A9 


14 


73 


13 


PS63 


16 


70 


14 


PS 16 


17 


68 


15 


A38 


14 


65 


21 


PS 65 


17 


60 


23 


A37 


24 


50 


26 


PS61 


31 


42 


27 



45 

38 



o Mean age at time of pretest was AO months (range: 4-64 months). 

o Racial/ethnic makeup of the group was 72 percent white, 20 
percent black, 5 percent Hispanic, and 3 percent Asian. 

o At least 20 percent of the children were from low-income families 
as measured by participation in Head Start. 

o "Language disordered" was the most frequent handicapping 
condition (43%), with multihandicapped the next most frequent 
subgroup (24%). 

o Sixty- three percent of the children were enrolled in public school 
programs. Of those attending private programs, 27 percent were 
enrolled in a program serving children with primary speech and/or 
language impairments. 

o Most children required a high level of intervention, with 64 
percent in Level 4 or 5 programs. 

o Three-fourths of the children received speech and language 
therapy, on the average, for one hour each week* 

o Children labeled as multihandicapped received a similar amount of 
speecn and language therapy but more occupational and physical 
therapy than children with other handicapping conditions. 

One of the major goals of the Preschool Evaluation Project is to address the 
issue of efficacy of the intervention which these preschool handicapped 
children are receiving. Despite several caveats which must be considered in 
interpretation of these data (the restricted amount of time between pre- and 
posttesting, the receipt by some children of special services prior to pre- 
testing, and the heterogeneity of the population), the following findings 
emerged: 

o With the exception of gross motor skills, Project children 
exhibi ted sta tis tically significant improvements in all areas of 
development beyond what would have been expected by maturation 
alone. 

o The largest gains were in the areas of personal-social and 
cognitive skills. 

o Children labeled as multihandicapped exhibited gains attributable 
to intervention in all areas of development except motor skills. 

o Children labeled as "articulation disordered" or "language 
disordered" made gains attributable to intervention, but not in 
speech and language areas. 

o An analysis of specific factors (e.g., number of minutes in 
therapy, race, and handicapping condition) which might be 
associated with progress or predictive of gain due to intervention 
showed no consistent trends. 
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o Parents reported overwhelming satisfaction with their preschool 
child's program and related services. 

o Service providers of Project children reportedly did not use BDI 
test results in educational plann-iug or discussions with parents 
about their child's educational progress. However, the majority 
(over 70%) considered BDI results as within plus or minus three 
months of their estimate of the child's functioning. 

o Reliance on parents as informants for BDI items related to school 
or group activities by and large resulted in responses similar to 
those provided by the child's teacher. 

These findings indicate that the gains these preschool handicapped children 
made during the 1983-84 school year were attributable to the services they 
received and beyond that which would have been expected with maturation 
alone. With an increased number of children in the Project population, a 
greater number of children in each handicapping classification, more 
extensive data on services received, and a longer time between pre- and 
posttesting, the data generated during Year 2 of the Project will be even 
more powerful. It is becoming clear that the results of this Project will 
provide answers to questions being asked across the country regarding the 
efficacy of intervention to preschool handicapped children. 
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APPENDIX A 
Documentation of Special Services 
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PROCRAM: 



WEEK OF: 



SERVICE DELIVER i FORM 
PERSON COMPLETING FORM: 



DATE: 




\A Service received (at least h scheduled time) 

J m Service NOT received 

Child or service dropped (specify reason) 
or service permanently changed 

EH|C 43 ****TIME RECORDED IN MINUTES**** 



Person accompanying child: 



P - Parent 

C - Other caregiver (grandparent, sitter, etc.) 
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APPENDIX B 

Parental Satisfaction with Service Provision 
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Preschool Evaluation Project 
Montgomery County Public Schools 
Rockville, Maryland 



SERVICE DELIVERY 
Parent Questionnaire 



Parent's Name: NOT . 

Often the educational services given to a child are *een differently by the many adults who work with that child. 
For example, the child's teacher may have a very different view of the speech therapy given to a child than the 
child's parents may have. The purpose of this questionnaire is to better understand parents* feelings and attitudes 
toward the service their child is receiving. 

Directions: To answer most questions, you will be asked to select an answer from a list of choices. Some questions 
will ask you to write out your answer. 

Example: 

How satisfied are you with the amount of the services your child receives? Indicate 
how satisfied you are with the amount of each service given to your child by writing 
the number of one of the answer choices in the space next to each service. 

Answer Choices: 

1=My child does not receive this service. 

2=Very satisfied. 

3=Satisfied. 

4=Neither satisfied nor dissatisfied. 
5= Dissatisfied. 
6= Very dissatisfied. 

Overall Program 3 

Physical Therapy (help in use of gross motor, sit, crawl, walk . . 1 



An answer of "3 M next to "Overall Program" means that you are satisfied with all 
parts of your child's program. An answer of "1" next to "Physical Therapy" means 
that you cannot rate physical therapy services because your child does not receive 
them. 

Your answers will be coded to an identification number (ID). All answers will be kept strictly confidential. 
Please use the reverse side of this page to clarify any of your answers. 



O PS 077^99338. revised 3/83 
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SERVICES DELIVERY Office Use 

Parent Questionnaire - _ . 

Form Code: 610 1-2 

Program ID: 4-5 

Parent ID: 6-c 

Program Name:_ 

1. How satisfied are you with the amount of services your child receives? Indicate how satisfied you are with 
the amount of each service given to your child by writing the number of one of the answer choices in tfe 
space next to each service. 

Answer Choices: 

1 =My child does not receive this service. 
2=Very satisfied. 
3= Satisfied. 

4—Neither satisfied, nor dissatisfied. 

5=DlssatisHed. 

6=Vory dissatisfied. 

Overall Program s 

Speech Therapy (help in speech, saying sounds and words, eating without choking) . . . -.o 

Occupational Therapy (help in use of fine motor, fingers, hands) ^^ 

Physical Therapy (help in use of gross motor, sit, crawl, walk) 12 

Behavior Management/Counseling (help with behavioral or emotional problems) 

Home Visits <u 



2. a. Approximately how many times during this school year has someone from your child's 

program visited and/or is scheduled to visit your home? Write your answer on the line to 

the right. If your answer is 2. write 02 on the line ie.*e 

b. On the average when staff have visited your home, how long did they stay? Write your answer in hou's 
and minutes in the space to the right. If your answer is 1 hour, write 1 on the line labeled Hours ' and CC 
on the !ine labeled "Minutes." 

Minutes tr.-.e 

Hours 19-2C 

3. How satisfied are you with the quality of the services your child receives? Indicate how 
satisfied you are with the quality of each service given to your child by writing the number of 
one of the answer choices in the space next to each service. 

Answer Choices: 

1 =My child does nol receivo this service. 

2= Very satisfied. 

3=Satlsfied. 

4=Nolther satisfied, nor dissatisfied. 
5= Dissatisfied. 
6=Very dissatisfied. 

Overall program 72 

Speech Therapy (help In speech, saying sounds and words, eating without choking) . . . 23 

Occupational Therapy (help in use of fine motor, fingers, hands) 

Physical Therapy (help in use of gross motor, sit, crawl, walk) 

Behavior Management/Counseling (help with behavioral or emotional problems) 

Home Visits . . 



2* 
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4. What do you like best about the services your child receives and why* 7 



5. What do you like least about the services your child receives and why? 



6. What recommendations do you have that you believe would improve the services your child receives? 



This concludes this questionnaire. Thank' you. 
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APPENDIX C 
BDI Questionnaire to Service Providers 
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Expiration date: 5/87 



Preschool Evaluation Project BATTELLE DEVELOPMENTAL 

Montgomery County Public Schools „ INVENTORY (BDI) QUESTIONNAIRE 

Rockville, Maryland 



If you have not bad sufficient contact with the BDI to answer the following 
questions, please check here and return the blank questionnaire. 



1) Approximately how many children that you serve have been tested with the 
BDI? (Please check one answer.) 

1-5 6-10 11-15 16-20 More than 20 



2) What percentage of the test booklets and results have you had an 
opportunity to review? (Please check one answer.) 

0-25% 26-50% 51-75% 76-100% 



For part A of the next five (5) questions, please use following scale: 

1 = BDI scores. are 6 or more months higher than I think they should be 

2 = BDI scores are about 3 months higher than I think they should be 

3 » BDI scores are about what I think they should be 

4 ■ BDI scores are about 3 months lower than I think they should be 

5 » BDI scores are 6 or more months lower than I think they should be 

6 = BDI scores are sometimes too high and sometimes too low, but 

generally about 6 months off 

7 « BDI scores are sometimes too high and sometimes too low, but 

generally about 3 months off 
9=1 cannot make a judgment 



general, what are your impressions of the BDI test results in the 
rsonal/Social Domain? (Please enter number from scale abovej 



. On what do you base your answer? (Rank order from 1 to 4, with 1 
being the source which most influenced your answer.) 

Clinical impressions/Classroom performance 

0 ther te s t data 

Feedback from other professionals 

Feedback from parents #### ' 
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4) A. In general, what are your impressions of the BDI test results in the 
Adaptive Domain? (Please enter number from scale above,) 



B. On what do you base your answer? (Rank order from 1 to 4.) 

Clinical impressions/Classroom performance 

Other test data , 

Feedback from other professionals 

Feedback from parents.. 



5) A. In general, what are your impressions of the BDI test results in the 
Motor Domain? (Please enter number from scale above.) 



B. On what do you base your answer? (Rank order from 1 to 4.) 

Clinical impressions/ Classroom performance ♦ 

Other test data 

Feedback c rom other professionals 

Feedback from parents 



6) A. In general, what are your impressions of the BDI test results in the 
Communication Domain? (Please enter number from scale above.) 

B. On what do you base your answer? (Rank order from 1 to 4.) 

Clinical impress! ons/Classroom performance 

Other test data 

Feedback from other professionals 

Feedback from parents 



7) A. In general, what are your impressions 0 f the BDI test results in the 
Cognitive Domain? (Please enter number from scale above.) 

B. On what do you base your answer? (Rank order from 1 to 4.) 

Clinical impressions/Classroom performance 

Other test data 

Feedback from other professionals 

Feedback from parents 



8) Did you attempt to use the BDI test results for educational planning? 
Yes No 

If yes, how useful were the BDI test results for this purpose? (Please 
circle the number which best describes your answer.) 

Very useful Moderately useful Not useful 

1 2 3 4 5 
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9) Did you attempt to use the BDI test results to talk with parents about 
their child's educational progress? 

Yes No 



If yes, how useful were the BDI test results for this purpose? (Please 

circle the number which best describes your answer,) 

Very useful . Moderately useful - Not useful 

1 2 3 45 

10) Please check one category which best describes your present role, 

Director /Coordinator 

Physical Therapist 

Special Educator/Teacher 

Speech/Language Pathologist 

Occupational Therapist 

Head Start Teacher 

Other (please specify) 



Additional Comments 



Thank you very much for your time. 
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APPENDIX D 

Additional Information About Value-added Analysi 
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The primary objective of the Preschool Evaluation Project is to determine 
the effectiveness of program participation for handicapped preschoolers. 
None of the standard evaluation approaches to program impact were feasible 
for the Project. There was no control group of unserved children. 
Furthermore, children could not be randomly assigned to programs; rather, 
children had to* be placed in the program most appropriate to their needs. 
On what basis could the Project determine that the programs had been 
effective? 

One approach which has been suggested in the early childhood special 
education literature for this kind of situation is the use of a "change 
index." Wolery and Bailey (1984) list a number of such change indices. The 
basic concept is to use what is known about the child from the pretest to 
project where the child should be functioning any number of months down the 
road. For example, a child who is 24 months old and functioning at the 12- 
month level prior to intervention would be projected to be functioning at 
the 18-month level at 36 months of age if all conditions stayed the same* 
If the child is functioning at a higher level, then the difference is 
attributed to an effective program. 

As Wolery (1983) points out, this kind of analysis assumes that the child's 
pretest score is an accurate reflection of the child's rate of development. 
To the extent that rates vary and the child is pretested at a particularly 
"slow" time, the projected rate of growth is inaccurately low. The child 
may show a substantial amount of gain between pre- and posttest due entirely 
to a developmental growth spurt. An analysis which uses a change index, 
however, will incorrectly attribute that growth to participating in a 
program. 

The particular type of change index selected for the Project data is based 
on "value-added" analysis (Bryk and Weisberg, 1976; Bryk, Strenio and 
Weisberg, 1980). The goal of this type of analysis is to determine the 
"value added" by the program above and beyond the growth that would have 
been expected without the program. The growth shown between the pre- and 
posttest is divided into two components: 

o The amount of growth due to maturation 
o The amount of growth due to program. 

Growth due to maturation is calculated by projecting where the children 
would have been expected to score based on the results of the pretesting. 
The value-added approach is not z* prone to error due to developmental 
growth spurts because the growth rates are computed for the entire group of 
children or subgroups of children through a regression equation rather than 
for each child individually through the use of a ratio. 

The basic steps in a simple value-added analysis are illustrated in the top 
of Figure D-L The particular analysis used by the Project was slightly 
more complicated in that it incorporated a number of other factors 
hypothesized to be related to developmental growth in the Project's 
population. These factors were the following: 

o Sex 

o Race/ethnic group (white, black, Hispanic, and other) 
o Family income (low income, not low income) 
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o Handicapping condition (speech and language impaired, multihandi- 
capped, other) 

o Severity level (as measured by program level, itinerant or full 
program) 

Based on their pretest scores, the analysis calculated a different growth 
rate for children who differed on these characteristics. For example, a 
black speech and language impaired male from a noa~low-income family 
receiving itinerant services had a different projected growth rate than a 
white multihandicapped female from a low-income-family in a full program. 
The use of the child's characteristics to compute different coefficients is 
illustrated in the bottom half of Figure B-l. 

The value-added approach provided a way to determine the extent of gains in 
each of the BDI domains which could be attributed to program participation. 
The amount of gain in each domain was then tested to see is it was 
significantly different from zero. 
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FIGURE D-l 



Steps in Value-Added Analysis 
Progress * Growth due to maturation + growth due to progras participation 

Step Example 
!• Regress pretest on age. Pretest 16 .8 Age 



*4 * 

*3« * M«Atn« 



2. Use coefficient from the regression, 
as the growth rate prior to pretest. 

3. Multiply rate by the amount of time 
in program to determine the amount of 
growth uue to maturation. 



Coefficient « .8 



.8 x 10 months « 8 points 



4. Subtract pretest from posttest to 
determine the total amount of growth. 

5. Subtract growth due to maturation from 
total growth to determine growth due to 
program participation. 



36 - 24 



12 



12-8-4 points 



Growth due to program participation « 4 points 



Computing Coefficients for Different Groups 



1. Regress pretest on age, sex, race, 

handicapping condition, family income 



Pretest « .8Age -.3Multi- 
handicapped + 
• IMale • . . 



2. Use "corrected coefficient** which 
depends or child's characteristics, 



Coef f icientMHtaale c 
.8 - .3 + .1 



.6 



Continue as above . 



Coefficient^™^- = 

.8 - .3 -.1 * .5 
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